Spatio-temporal Embedding for Statistical Face Recognition from Video

This paper addresses the problem of how to learn an appropriate representation from video to benefit video-based face recognition. We pose it as learning spatio-temporal embedding (STE) from raw video. STE of a video sequence is defined as its condensed version capturing the essence of space-time characteristics of the video. Relying on co-occurrence statistics of training videos, Bayesian keyframe learning leads to the temporal embedding, keyframes, of each video. Given supervised signatures of face videos, nonparametric discriminant embedding (NDE) learned from the keyframes makes up STE. A statistical formulation in terms of STEs to the video-based recognition problem. Spatial Embedding: NDE Experimental results